A Connection Between One-Step Rl And Critic Regularization In Reinforcement Learning